The use of web-scraped data to analyse the dynamics of clothing and footwear prices

Adam Juszczak

doi:https://doi.org/10.59139/ws.2023.09.2

Adam Juszczak Uniwersytet Łódzki, Wydział Ekonomiczno-Socjologiczny, Polska / University of Lodz, Faculty of Economics and Sociology, Poland ORCID: https://orcid.org/0000-0001-6027-6605 Wiadomości Statystyczne. The Polish Statistician, vol. 68, 2023, 9, s. 15-33 Opublikowano online: 29 września 2023 DOI https://doi.org/10.59139/ws.2023.09.2 Sposób cytowania: Juszczak, A. (2023). The use of web-scraped data to analyse the dynamics of clothing and footwear prices. Wiadomości Statystyczne. The Polish Statistician, 68(9), 15–33. https://doi.org/10.59139/ws.2023.09.2.

379 Wyświetlenia 82 Pobrania

ARTYKUŁ

(Angielski) PDF

STRESZCZENIE

Web scraping is a technique that makes it possible to obtain information from websites automatically. As online shopping grows in popularity, it became an abundant source of information on the prices of goods sold by retailers. The use of scraped data usually allows, in addition to a significant reduction of costs of price research, the improvement of the precision of inflation estimates and real-time tracking. For this reason, web scraping is a popular research tool both for statistical centers (Eurostat, British Office of National Statistics, Belgian Statbel) and universities (e.g. the Billion Prices Project conducted at Massachusetts Institute of Technology). However, the use of scraped data to calculate inflation brings about many challenges at the stage of their collection, processing, and aggregation. The aim of the study is to compare various methods of calculating price indices of clothing and footwear on the basis of scraped data. Using data from one of the largest online stores selling clothing and footwear for the period of February 2018–November 2019, the author compared the results of the Jevons chain index, the GEKS-J index and the GEKS-J expanding and updating window methods. As a result of the calculations, a high chain index drift was confirmed, and very similar results were found using the extension methods and the updated calculation window (excluding the FBEW method).

SŁOWA KLUCZOWE

inflation, web scraping, online shopping, GEKS-J

JEL

C43, C49

BIBLIOGRAFIA

Auer, J., & Boettcher, I. (2017). From price collection to price data analytics. How new large data sources require price statisticians to re-think their index compilation procedures. Experiences from web-scraped and scanner data. Ottawa Group. https://www.ottawagroup.org/Ottawa/ottawagroup.nsf/4a256353001af3ed4b2562bb00121564/1ab31c25da944ff5ca25822c00757f87/$FILE/From%20price%20collection%20to%20price%20data%20analytics%20-Josef%20Auer,%20Ingolf%20Boettcher%20-Paper.pdf.

Białek, J. (2021). PriceIndices – a New R Package for Bilateral and Multilateral Price Index Calculations. Statistika. Statistics and Economy Journal, (2), 122–141. https://www.czso.cz/documents/10180/143550797/32019721q2_bialek.pdf/3cd5bf11-22f4-4ee5-b294-1d7d5909e4b4?version=1.1.

Białek, J., & Bobel, A. (2019). Comparison of Price Index Methods for the CPI Measurement Using Scanner Data. https://www.ottawagroup.org/Ottawa/ottawagroup.nsf/home/Meeting+16/$FILE/Comparison%20of%20Price%20Index%20Methods%20paper.pdf.

Białek, J., Kłopotek, M., & Panek, T. (2022). Nowoczesne technologie i nowe źródła danych w pomiarze inflacji. Główny Urząd Statystyczny. https://bws.stat.gov.pl/BWS/gus_bws_70_nowoczesne_technologie_i_nowe_zrodla_danych_w_pomiarze_inflacji.pdf.

Bitner, T., & Stech, G. (2019, April 9). GUS: Big Data to nasz priorytet. https://www.computerworld.pl/wywiad/GUS-Big-Data-to-nasz-priorytet,412891.html.

Bosch, O. (n.d.). Uses of web scraping for official statistics. ESTP course on Big Data Sources – Web, Social Media and Text Analytics. https://circabc.europa.eu/sd/a/5e250346-44a9-471b-87f1-5b5ddb59aa77/1_Big%20Data%20Sources%20part3-Day%201-A%20Use.pdf.

Chessa, A. G. (2016). A new methodology for processing scanner data in the Dutch CPI. Eurostat Review of National Accounts and Macroeconomic Indicator, (1), 49–69. https://ec.europa.eu/eurostat/cros/system/files/euronaissue1-2016-art2.pdf.

Chessa, A. G. (2021). A Product Match Adjusted R Squared Method for Defining Products with Transaction Data. Journal of Official Statistics, 37(2), 411–432. https://doi.org/10.2478/jos-2021-0018.

Chessa, A. G., & Griffioen, R. (2019). Comparing Price Indices of Clothing and Footwear for Scanner Data and Web Scraped Data. Economie et Statistique. Economics and Statistics, (509), 49–68. https://doi.org/10.24187/ecostat.2019.509.1984.

Chuanyang, F., & Lee Wen Hao, J. (2016). Experiences with the Use of Online Prices in Consumer Price Index. Statistics Singapore Newsletter, (2), 1–4. https://www.singstat.gov.sg/-/media/files/publications/economy/ssnsep16-pg1-4.ashx.

Eltetö, Ö., & Köves, P. (1964). Egy nemzetközi összehasonlításoknál fellépö indexszámításl. Statisztikai Szemle, 42, 507–518. http://real-j.mtak.hu/21330/5/Statisztikai_szemle_1964_05.pdf.

Eurostat. (2020). Practical guidelines on web scraping for the HICP. https://ec.europa.eu/eurostat/documents/272892/12032198/Guidelines-web-scraping-HICP-11-2020.pdf/.

Gini, C. (1931). On the Circular Test of Index Numbers. Metron, 9(9), 3–24.

de Haan, J., & van der Grient, H. A. (2011). Eliminating chain drift in price indexes based on scanner data. Journal of Econometrics, 161(1), 36–46. https://doi.org/10.1016/j.jeconom.2010.09.004.

de Haan, J., Willenborg, L., & Chessa, A. (2016). A Review of Price Index Methods for Scanner Data. https://doi.org/10.13140/RG.2.2.27350.88643.

Jevons, W. S. (1865). On the Variation of Prices and the Value of the Currency since 1782. Journal of the Statistical Society of London, 28(2), 294–320. https://doi.org/10.2307/2338419.

Juszczak, A. (2021). Zastosowanie danych scrapowanych w pomiarze dynamiki cen. Acta Universitatis Lodziensis. Folia Oeconomica, 1(352), 25–37. https://doi.org/10.18778/0208-6018.352.02 .

Lamboray, C. (2017). The Geary Khamis index and the Lehr index: how much do they differ?. 15th Meeting of the Ottawa Group, Elville am Rhein. https://www.ottawagroup.org/Ottawa/ottawagroup.nsf/4a256353001af3ed4b2562bb00121564/1ab31c25da944ff5ca25822c00757f87/$FILE/The%20Geary%20Khamis%20index%20and%20the%20Lehr%20index%20how%20much%20do%20they%20differ%20-%20Claude%20Lamboray%20-%20Presentation.pdf.

van Loon, K., & Roels, D. (2018, May 7–9). Integrating big data in the Belgian CPI. Meeting of the Group of Experts on Consumer Price Indices, Geneva. https://unece.org/fileadmin/DAM/stats/documents/ece/ces/ge.22/2018/Belgium.pdf.

Lünnemann, P., & Wintr, L. (2006). Are Internet Prices Sticky? (ECB Working Paper No. 645). https://www.ecb.europa.eu/pub/pdf/scpwps/ecbwp645.pdf.

Macias, P., & Stelmasiak, D. (2019). Food inflation nowcasting with web scraped data (NBP Working Papers No. 302). https://static.nbp.pl/publikacje/materialy-i-studia/302_en.pdf.

Netcomm Suisse Observatory, United Nations Conference on Trade and Development. (2020). COVID-19 and E-commerce Findings from a survey of online consumers in 9 countries. https://unctad.org/system/files/official-document/dtlstictinf2020d1_en.pdf.

Persson, E. (2019). Evaluating tools and techniques for web scraping [Second cycle degree project, KTH School of Electrical Engineering and Computer Science]. https://www.diva-portal.org/smash/get/diva2:1415998/FULLTEXT01.pdf.

Polidoro, F., Giannini, R., Conte, R. L., Mosca, S., & Rosetti, F. (2015). Web scraping techniques to collect data on consumer electronics and airfares for Italian HICP compilation. Statistical Journal of the IAOS, 31(2), 165–176. https://doi.org/10.3233/sji-150901.

Radzikowski, B., & Śmietanka, A. (2016). Online CASE CPI. 1st International Conference on Advanced Research Methods and Analytics, Valencia. http://dx.doi.org/10.4995/CARMA2016.2016.3133.

Szulc, B. (1964). Indices for multiregional comparisons. Przegląd Statystyczny, (3), 239–254.

Wróć do: